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THEORETICAL  RESEARCH 
OBJECTIVE 

The  goal  of  this  work  is  to  characterize  mathematically  the  essential 
mechanisms  and  principles  of  operation  of  the  mamalian  olfactory  neural 
network  and  evaluate  its  computation  and  pattern  recognition  capabilities. 
The  intent  is  to  explicate  novel  design  principles  that  may  underly  the 
superior  performance  of  biological  systems  in  pattern  recognition  through 
detailed  study  of  a  particular  system.  This  research  will  be.  carried  out  in 
collaboration  with  the  systems  theory  group  of  the ,Berkel^»y Electrical 
Engineering  department  headed  by  Shankar  Sastry  and  Pravin  Varaiya  who  are 
already  funded  by  the  Air  Force  JSEP  program^for  the  purpose  of  estab¬ 
lishing  a  theoretical  framework  for  the  evaluation  of  architectures  and 
algorithms  for  parallel  computation  -  with  particular  emphasis  on  neural 

networks  00  ^ 

c"  ■  '  cu>  •  5 r')  - 


APPROACH 

The  specific  strategy,  in  line  with  the  program  of  the  systems 
group,  is  to  carry  out  a  study  of  the  pattern  recognition  dynamics  of  a 
simplified  version  the  model  of  the  first  olfactory  sensory  cortex  develop¬ 
ed  by  Freeman  and  shown  to  accurately  simulate  the  observed  temporal 
behavior  of  the  olfactory  EEG.  This  will  involve  detailed  application  of 
the  numerical  and  analytic  tools  of  dynamical  systems  theory,  and  bifurca¬ 
tion  theory  in  particular  -  both  center  manifold  theory  and  the  newly 
emerging  singularity  theory  of  bifurcations  with  symmetry  -  in  an  effort  to 
obtain  the  deepest  possible  mathematical  insight  into  the  intrinsic 
mechanisms  of  pattern  recognition  and  learning.  Particular  examination  will 
be  made  of  the  functional  importance  of  the  unusual  network  features 
contained  in  the  natural  design  of  this  system  -  bifurcation,  oscillation, 
chaos,  and  sigmoid  gain  variation. 

CONTRIBUTION 

This  laboratory  is  in  a  unique  position  to  exploit  the  historically 
fertile  interaction  between  concepts  of  natural  and  artificial  design, 
since  this  is  the  only  laboratory  to  have  demonstrated  correlations  between 
spatial  patterns  of  cortical  network  activity  and  the  pattern  recognition 
behavior  of  an  animal.  The  olfactory  system  is  one  of  the  best  of  the 
candidates  presently  being  investigated  for  a  biological  system  which  may 
actually  function  as  an  "associative  memory"  according  to  the  general 
principles  of  current  engineering  "neural  networks".  This  allows  both  the 
evaluation  of  the  biological  validity  of  present  theoretical  network 
constructs,  and  the  generation  of  new  principles  from  the  experimental 
observation  of  a  real  neural  network.  The  biological  experimentation  is 
encouraging  us  to  study  the  properties  of  networks  with  continuous  dynamics 
in  layers  with  crosscoupling  feedback.  There  is  little  work  to  date 
emphasizing  the  special  pattern  recognition  capabilities  of  feedback  nets 
with  smooth  nonlinearities.  In  addition,  the  biological  net  inspires 
network  models  with  properties  that  are  quite  foreign  to  any  present 
engineering  effort.  The  net  is  driven  in  and  out  of  its  pattern  recognition 
mode  through  a  bifurcation  and  works  with  oscillatory  and  possibly  chaotic 
attractors . 
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RELEVANCE 

The  neural  network  model  under  investigating  may  be  considered  a 
nonsymmetrically  coupled  variant  of  the  model  of  analog  neural  dynamics 
popularized  by  Hopfield.  These  investigations  will  thus  be  relevant  to  the 
mainstream  of  neural  network  research.  The  challange  of  achieving  a 
theoretical  understanding  of  this  real  biological  situation  poses  many 
problems  of  mathematical  analysis  which  are  in  fact  on  or  beyond  the 
present  frontier  of  network  research.  Examination  of  the  complex  dynamics 
and  advantages  of  nonsymetrically  coupled  nets  is  already  cited  as  a 
research  goal  by  the  systems  group.  The  prominance  of  bifurcation  and 
oscillatory  and  possibly  chaotic  dynamics  observed  in  the  olfactory  system, 
and  the  need  to  explore  learning  algorithms  for  the  storage  of  attracting 
trajectories  in  order  to  model  its  hypothesized  operation,  places  these 
efforts  on  the  frontier  of  network  research.  In  the  other  direction,  study 
can  be  made  of  the  biological  validity  in  this  experimental  system,  and  the 
impact  on  the  pattern  recognition  performance  of  this  model  of  concepts 
already  developed  in  the  network  literature  such  as  adaptive  resonance, 
hidden  units, 

and  higher  order  correlations. 
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ACCOMPLISHMENTS 

NORMAL  FORM  POJECTION  ALGORITHM 


Introduction 

A  new  class  of  learning  algorithms  for  the  storage  of  static  and 
periodic  attractors  in  recurrent  analog  neural  networks  has  been  developed. 
The  method  allows  programming  of  the  ambient  vector  field  independent  of 
the  patterns  to  be  stored.  The  stability  of  cycles  and  equilibria, 
geometry  of  basins  of  attraction,  rates  of  convergence  to  attractors,  and 
the  location  in  parameter  space  of  primary  and  secondary  bifurcations  can 
be  programmed  in  a  prototype  vector  field  -  the  normal  form.  Fixed  points 
on  the  axes  of  this  system  of  nonlinear  ordinary  differential  equations  are 
then  linearly  transformed  into  desired  spatial  or  spatio-temporal  patterns 
by  projecting  the  system  into  network  coordinates  -  the  standard  basis  - 
using  the  desired  vectors  as  columns  of  the  transformation  motrix.  This 
method  of  network  synthesis  is  roughly  the  inverse  of  the  usuol  proceedure 
in  bifurcation  theory  for  analysis  of  a  given  physical  system.  It  differs 
from  the  numerical  approach  of  previous  papers  (Baird  88)  in  that  the  last 
stage  of  analysis,  the  calculation  of  normal  form  coeficients,  is  inverted 
as  well. 

In  the  general  case,  the  resulting  network  has  forth  order  correla¬ 
tions,  but  the  use  of  restrictions  on  the  detail  of  vector  field  program¬ 
ming  and  the  types  of  patterns  to  be  stored  result  in  network  architectures 
requiring  only  second  order  correlations.  The  network  can  be  constructed 
with  the  usual  crossbar  interconnections  by  implementing  the  higher  order 
interactions  within  the  units.  For  biological  modeling,  where  possibly  the 
patterns  to  be  stored  are  sparse  and  nearly  orthogonal,  the  learning  rule 
for  periodic  patterns  becomes  a  complex  outer  product  rule  which  is  local, 
additive,  commutative,  and  incremental.  It  reduces  to  the  usual  Hebb-iike 
rule  for  static  attractors.  Networks  with  excitatory  and  inhibitory  units 
that  approximate  known  biological  architectures  such  as  the  olfactory  bulb 
and  cortex  can  also  be  constructed  by  proper  choice  of  constraints. 

Standing  or  traveling  waves  may  be  stored  to  mimic  the  different  patterns 
of  neurophysiological  activity  observed  in  the  olfactory  system. 

Although  developed  from  the  bifurcation  theory  approach  discussed  in 
previous  papers  (Baird  88),  the  results  on  these  networks  are  global  and 
exact.  They  are  not  restricted  to  the  neighborhood  of  any  bifurcation,  and 
can  be  discussed  without  referance  to  bifurcation  theory.  There  is  however 
a  multiple  bifurcation  in  each  system  at  critcal  values  of  different 
bifurcation  parameters  that  can  be  used  to  generate  the  attractors  from  a 
single  equilibrium  at  the  origin.  The  projection  algorithm  provides  one 
solution  to  the  problem  of  storing  exact  analog  fixed  points  in  a  neural 
network  -  at  a  capacity  of  2N  attractors  for  an  N  node  network.  Periodic 
attractors  which  are  simple  cycles  can  be  stored  at  a  capacity  of  N/2,  and 
arbitrarily  complicated  periodic  attractors  can  probably  be  constructed 
from  any  subset  of  these  by  a  kind  of  Fourier  synthesis.  If  the  stability 
of  fixed  points  on  some  axes  of  the  normal  form  can  be  transfered  to  an 
interior  fixed  point  by  vector  field  programming,  then  the  temporal  pattern 
in  the  network  corresponding  to  this  "mixed  mode"  is  a  linear  combination 
of  the  axis  patterns  at  their  different  frequencies  .  The  vectorfield  can 
be  programmed  so  that  the  axis  attractors  are  the  only  attractors  in  the 
system  -  there  are  no  "spurious  attractors".  If  symmetric  normal  form 
coeficients  are  chosen  to  program  the  vector  field,  then  the  normal  form 
becomes  a  gradient  vector  field.  It  is  excactly  the  gradient  of  an 
explicit  potential  function  which  is  also  a  strict  Liapunov  function  for 
the  system. 
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When  there  are  symmertries  chosen  for  the  normal  form  equations  , 
then  the  network  equations  are  equivariant,  and  the  network  attractors  are 
interrelated  by  a  group  of  transformations .  These  are  given  from  the 
learning  process  even  though  there  are  no  restrictions  on  the  patterns 
being  learned.  With  such  symmetries,  the  basins  of  attraction  can  be  made 
into  hyperplane  bounded  wedges.  These  hyperplane  basin  boundaries  can  be 
positioned  by  the  learning  algorithm  to  realize  optimal  clustering  of 
inputs  into  categories  and  retrieval  by  nearest  Euclidean  distance.  This 
is  an  analog  network  which  can  be  restricted  to  categorize  like  a  binary 
network  of  linear  threshold  logic  units,  and  still  produce  a  specified 
analog  output.  While  analog  behavior  can  usually  be  approximated  by  a 
sufficiently  high  dimensional  digital  system,  that  increased  dimensionality 
is  one  of  the  prices  paid. 

An  autoassociative  memory  like  this  one  can  always  be  used  as  a 
heteroassociative  memory  by  wiring  input  to  some  of  the  components  of  the 
state  vector  and  taking  output  from  the  rest.  Because  the  basins  of 
attraction  are  known  exactly  in  this  system,  it  will  be  possible  to  explore 
in  detail  its  mapping  and  generalization  behavior  when  it  is  used  for 
heteroassociation.  The  work  of  James  Anderson  demonstrates  that  almost  all 
of  the  applications  of  the  feedforward  backpropagatlon  networks  can  be 
approached  as  well  with  a  feedback  network  used  in  this  fashion.  There  is 
however  little  mathematical  analysis  of  these  systems,  and  we  expect  that 
analysis  of  the  present  networks  may  reveal  superior  generalization 
performance  over  feedfoward  networks  in  many  task  domains. 

A  unique  and  important  application  of  the  projection  algorithm  is  to 
construct  networks  for  the  exploration  of  the  computational  capabilities  of 
more  exotic  vector  fields  involving,  for  example,  quasiperiodic  or  chaotic 
flows.  A  neural  network  can  be  synthesized  to  realize  any  nonlinear 
behavior  that  is  known  in  the  catalogue  of  well  analysed  normal  forms.  The 
normal  form  theorem  asserts  that  there  is  a  large  class  of  ode’s  and  pde’s 
which  give  local  vector  fields  that  are  topologically  equivalent  to  that  of 
any  normal  form  under  consideration.  This  implies  that  there  are  many 
other  possible  network  architectures  to  be  discovered  that  can  realize  the 
performance  of  that  produced  by  the  projection  algorithm. 


Projection  Algorithm 

fundamental  Theorem 

The  central  result  is  most  compactly  stated  as  the  following  theor¬ 
em:  Any  set  S  , 

i(0s  +  ws) 

s  »  1,2,...,N/2  ,  of  cycles  rs  xs  e  of  linearly  lndepen 

dent  vectors  of  relative  component  amplitudes  x*  of  RN  and  phases  0s  of 
Sn  ,  with  frequencies  ws  of  R  and  global  amplitudes  rs  of  R,  may  be 
established  in  the  vector  field  of  the  analog  third  order  network: 


xi  >  -  t  xi  +  Ej  TiJ  xj  +  Ejki  Tijkl  xj  xk  xl  +  dl  d(t)  (3.1) 

by  some  variant  of  the  learning  operation  : 


^lj  *  Emn  Pim  Jim  P-lnj  ,  Tijkl  *  Emn  Plm  Aim  P-lmJ  P-Ink  P-lnl  (3.2) 


5 


.where  the  NxN  matrix  P  contains  the  real  and  imaginary  components 

10s 

[  xs  cos  0s  ,  xs  sin  0s  ]  of  the  complex  eigenvectors  x  e  as 
columns,  J  is  an  NxN  matrix  of  complex  conjugate  eigenvalues  in  diagonal 

blocks,  and  is  an  NxN  matrix  of  2x2  blocks  of  repeated  coeficients 
of  the  normal  form  equations.  The  vector  field  of  the  dynamics  of  the 
global  amplitudes  rs  and  phases  9s  is  given  exactly  by  the  normal  form 
equations  ; 


rs  ■  us  rs  -  rs  Ej  asj  rj2 


®s  -  ws  +  EJ  bsj  rj2 


(3.3) 


In  particular,  for  ask  >  0  ,  and  ass/aks  <  1  ,  for  all  s  and  k,  the  cycles 
s  =  1,2,...,N/2  are  stable,  and  have  amplitudes 

rs  »  ( us/ass  )l/2  ,  where  us  «  1  -  t  . 


Proof 

The  proof  of  this  is  instructive  since  it  is  a  constructive  proof. 
An  informal  version  proceeds  by  showing  first  that  there  are  always  fixed 
points  on  the  axes  of  these  amplitude  equations,  whose  stability  is  given 
by  the  coeficients  of  the  nonlinear  terms.  Then  the  network  above  is 
constructed  from  these  equations  by  two  coordinate  transformations.  The 
first  is  from  polar  to  Cartesian  coordinates,  and  the  second  is  a  linear 
transformation  from  cannonical  "mode"  coordinates  into  the  standard  basis 


ei,  e2 . eN  .  or  "network  coordinates".  This  second  transformation 

constitutes  the  ‘learning  algorIthm,',  because  it  transforms  the  simple 
fixed  points  of  the  amplitude  equations  into  the  specific  spatio-temporal 
memory  patterns  desired  for  the  network . 


Amplitude  fixed  points 

Because  the  amplitude  equations  are  independent  of  the  rotation  ®  , 
the  fixed  points  of  the  amplitude  equations  characterize  the  assymptotic 
states  of  the  underlying  oscillations.  The  stability  of  these  cycles  is 
therefore  given  by  the  stability  of  the  fixed  points  of  the  amplitude 

equations.  On  each  axis  rs  <  the  0ther  components  rj  are  zero,  by  defini¬ 
tion,  hence 


rJ  -  rj  (  uj  -  Ek  ajk  rk2  )  -  o  ,  for  rj  -  o  ,  which  leaves 


rs  =  rs  (  us  -  ass  rs2  )  ,  and  rs  «  o  ,  when  rs2  »  us/ass  .  Thus 
we  have  an  equilibrium  on  each  axis  s  ,  at  rs  ,  (us/ass)i/2  as  claimed. 
Now  the  Jacobian  of  the  amplitude  equations  at  some  fixed  point  r “  has 


elements 
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n 

Jlj  -  -  2  alj  r~  1  r~J  ,  Jll  =  ui  -  3  oil  r~  12  -  EJ  alj  r.*H  (3.4) 

J  /  -  i 

For  a  fixed  point  r  s  on  Qxis  s  ,  Jlj  ■  o  ,  since  r~i  or  r~j  *  o  ,  making 
J  a  diagonal  matrix  whose  entries  are  therefore  its  eigenvalues.  Now 

ui  -  als  r~s2  ,  for  i  /=  s  ,  and  Jss  »  us  -  3  ass  r~s2  .  Since  r~s2  » 
us/ass  ,  Jss  ■  -  2  us  ,  and  Jil  =  ui  -  als  (us/ass)  .  This  gives  ais/ass  > 
ui/us  as  the  condition  for  all  negative  eigenvalues  that  assures  the 
stability  of  r  s  Choice  of  aji/ail  >  uj/ui,  for  all  i,J  ,  therefore 
guarantees  stability  of  all  axis  fixed  points. 

Coordinate  transformations 


We  now  construct  the  neural  network  from  these  well  behaved 
equations  by  the  following  transformations, 

First;  polar  to  Cartesian  ,  (rs0s)  to  (v2s.1<v2s)  .  Uslng 

v2s-i  =»  rs  cos  0s  ,  v2s  ■  rs  sin  0s  ,  and  differentiating  these  gives, 

v2s-l  -  rs  cos  0s  -  rs  sin  0s  0s  , 

v2s  -  rs  sin  0s  +  rs  cos  0s  0s  , 


by  tho  chain  rule.  Now  substituting  cos  0S  _  V2S-i/rs  and 
rs  sin  0s  «  v2s. 


gives : 


/2s-l  »  v2s-i/rs  rs  -  v2s  0s 


'2s 


■  v2s  rs  +  v2s-i/rs  0s 
Entering  the  expressions  of  the  normal  form  for  rs  and  0S  gives: 

v2s-i  »  v2s-i/rs  (us  rs  +  rs  EJ  asj  rj2)  -  v2s  (ws  +  Ej  bsj  rj2) 
and  since  rj2  ,  V2s-12  +  v2s2  , 


n/2 

v2s-1  »  us  v2s-1  -  ws  v2s  +  Ej  [v2s-1  as)  -  v2s  bsj]  (v2J-12  +  v2)2) 
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Similarly, 


n/2 

v2s  «  us  v2s  +  ws  v2s-1  +  EJ  [v2s  asj  +  v2s-l  bsj]  (v2J-12  +  v2J2) 

Setting  the  bsj  .  0  fQr  simplicity,  choosing  us  »  -  t  +  1  to  get  a 

standard  network  form,  and  reindexing  1,J  ■  1,2 . N  ,  we  get  one 

Cartesian  equivalent  of  the  polar  normal  form  equations. 

n  n 

vi  -  -  t  vl  +  Ej  Jij  vj  +  vi  Ej  AiJ  vj2  (3.4) 


Here  J  is  a  matrix  containing  2x2  blocks  along  the  diagonal  of  the 
local  couplings  of  the  linear  terms  of  each  pair  of  the  previous  equations 

v2s-i  ,  v2s  ,  with  -  t  separated  out  of  the  diagonal  terms.  The  matrix  A 


has 

2x2  blocks 

of  identical 

coeficlents 

Of 

the  nonlinear 

terms 

each 

pair . 

i  1  -  w1 

1 

an 

ail 

ai2 

ai2 

l 

« 

! 

1  w,  1 

1, 

.  ! 

an 

ail 

ai2 

ai2 

1 

t 

J  » 

1 

! 

1 

1  -  w2 

1 

A  » 

1 

Q21 

021 

a22 

022 

1 

1 

• 

1 

1 

1 

! 

! 

w2  , 

1 

021 

021 

022 

022 

1 

1 

« 

Learning  Transformation  -  Linear  Term 


J  can  be  viewed  as  the  cannonical  form  of  a  real  matrix  with  complex 
conjugate  eigenvalues,  where  the  conjugate  pairs  appear  in  blocks  along  the 
diagonal  as  shown.  The  equations  (3.4)  describe  the  Interaction  of  these 
linearly  uncoupled  conjugate  node  pairs  due  to  the  coupling  of  the  non¬ 
linear  terms.  They  can  be  viewed  as  local  oscillators  with  frequencies 

chosen  as  the  w,  anc|  nqnlinear  comaetition  chosen  by  the  ast.  (shunting 
eqns?)  The  second  coordinate  transformation  introduces  further  off  diagonal 
coupling  of  the  linear  terms  so  that  each  local  oscillator  becomes  a 
desired  spatially  distributed  pattern  or  "mode".  We  can  intertpret  the 
normal  form  equations  as  network  equations  in  eigenvector  or  "memory 
coordinates".  In  other  words,  we  can  think  of  them  as  having  been  produced 
from  some  network  by  a  diagonalizing  coordinate  transformation  P  ,  so  thot 

each  axis  pair  v2j  v2.  ls  „  compiex  eigenvector  basis,  and 
the  complex  conjugate  eigenvalues  of  some  coupling  matrix  T 
coordinates.  In  this  case,  for  the  linear  term, 


1* 


P-1  T  P 


Then  it  is  clear  that 


P  J  P-i 


The  matrix  P  is  usually  constructed  of  columns  which  are  the  eigenvectors 
calculated  from  the  matrix  T  to  be  diagonalized.  For  the  present 
purpose,  we  work  backwards,  choosing  eigenvectors  for  P  and  eigenvalues 


8 


for  J  ,  then  constructing  T  by  the  formula  above.  To  satisfy  the 
the'rem,  we  choose  as  columns  the  real  and  imaginary  vectors 
r  xs  cos  Os  ,  x  sin  Os  ]  of  the  cycles  x  elOs  of  the  set  S  to  be  learn¬ 
ed.  Any  linearly  independent  set  of  complex  eigenvectors  in  P  can  be 
chosen  to  transform  the  system  and  become  the  patterns  "stored"  in  the 
vectorfield  of  network  dynamics,  (diagonalizatlon?)  This  "learning"  process 
might  also  be  thought  of  as  giving  a  vector  "representation"  to  "Platonic" 
oscillators  in  idealized  coodinates  by  "projecting"  out  of  that  space  onto 
the  vectors  of  the  standard  basis  -  hence  the  name  "projection  algorithm". 
The  transformation  exchanges  simple  vectors  or  "spatial  patterns"  in  a 
complicated  coordinate  basis  for  complicated  patterns  in  a  simple  coor¬ 
dinate  basis.  If  we  write  the  matrix  expression  for  T  above  in  component 
form,  we  recover  the  expression  given  in  the  theorem  for  T^ 

^ij  ■  Emn  Pirn  Jon  P-lnj 
Nonlinear  term  projection 

The  nonlinear  terms  are  transformed  as  well,  but  the  expression 
cannot  be  easily  written  in  matrix  form.  Using  the  component  form  of  the 
transformation , 

xi  »  EJ  PlJ  vj  ,  xi  »  EJ  PiJ  vj  vj  »  Ek  P-ljk  xk  ,  and 

substituting  into  (3.4),  gives: 


xi  -  (-t+1)  EJ  PiJ  (Ek  P-IJk  xk)  +  Ej  PiJ  Ek  Jjk  (El  P-lkl  xl) 

+  EJ  PIJ  (Ek  P-1  Jk  xk)  El  AJ1  (En  P-Ilm  xi»)  (En  P-Iln  xn) 

Rearranging  the  orders  of  summation  gives, 

X1  -  (-t+1)  Ek  (Ej  PiJ  P-IJk )  xk  +  El  (Ek  EJ  PlJ  Jjk  P-lkl)  xl 
+  E«  Ek  (El  EJ  PiJ  P-IJk  AJ1  P-Ilm  P-Iln)  xk  xn  xn 

Finally,  performimg  the  bracketed  summations  and  relabeling  indices 
gives  us  the  network  (3.1.1)  of  the  theorem, 

xi  -  -  t  xi  +  EJ  TiJ  xj  +  Ejkl  Tljkl  xj  xk  xl 
with  the  expression  for  the  tensor  of  the  nonlinear  term, 


^ijkl  »  Emn  Pin  Ann  P-lnj  P-Ink  P-lnl 


O.E.D. 


A  Hardware  Architecture  of  a  Neural  Network  for 
Pattern  Recognition  Following  Learning 


We  designed,  built  and  tested  an  electronic  neural  network  for  class¬ 
ification  of  patterns  that  are  defined  with  a  "learning”  algorithm.  The  non¬ 
linear  dynamics  of  the  network  are  motivated  by  experimental  findings  in  EEG 
recordings  of  the  olfactory  bulb.  These  findings  indicate  that  a  massively 
parallel  architecture  can  be  utilized  to  best  describe  the  bulb’s  dynamics.  The 
electronic  design  is  a  digital/analog  hybrid  approach,  utilizing  the  speed  and 
flexibility  of  random  access  memory  (RAM)  for  the  storing  of  synaptic  strengths 
that  are  modified  in  "learning",  while  still  preserving  the  analog  computational 
power  of  neural  networks.  This  approach  also  includes  a  multiplexing  scheme 
which  decreases  the  number  of  connections,  in  the  hordware,  from  order  N2  to  or¬ 
der  N,  where  N  is  the  number  of  neural  units.  The  key  operation  is  a  bias  thst 
is  induced  by  an  step  input,  causing  the  system  to  switch  from  a  low-gain  equi- 
libium  state  to  a  high-gain  limit  cycle  state.  The  output  is  read  as  a  spatial 
pattern  of  oscillitory  amplitudes. 


Central  Pattern  Generating  and  Recognizing  in  Olactory  Bulb: 
A  Correlation  Learning  Rule 


A  learning  rule  called  an  input  correlation  rule  that  can  simplify  explora¬ 
tion  of  the  behavior  of  learning,  generating,  and  classifying  patterns  in  the 
vertebrate  olfactory  system  is  proposed.  We  apply  this  correlation  rule  to  a  set 
of  fully  Interconnected  coupled  oscillators  that  comprises  a  dynamical  model  of 
the  olfactory  bulb  so  os  to  form  "templates"  of  oscillators  with  strengthened  in¬ 
terconnection  in  respect  to  inputs  classed  as  "learned."  We  obtain  a  content  ad¬ 
dressable  memory  in  which  phase  coherent  oscillation  provides  for  central  pattern 
generation  and  recognition.  We  use  this  analog  model  neural  network  to  simulate 
dynamic  features  of  the  olfactory  bulb  in  detail  by  numerical  integration  and 
multivariate  analysis.  The  model  classifies  100£  correctly  for  incomplete  in¬ 
puts,  testing  inputs  about  their  training  centroids,  and  distortion  by  noise  that 
is  defined  as  input  to  nontemplate  elements.  The  model  also  allows  substantially 
overlapping  templates,  which  implies  that  it  possesses  a  large  information 
capacity.  For  multiple  inputs  the  model  gives  correct  output  of  the  forms  A  and 
B,  A  and  not  B,  B  and  not  A,  or  neither.  The  initial  conditions  of  the  model  at 
the  time  of  onset  of  input  play  no  role  in  classifcation .  Classification  is 
achieved  within  20  to  50  ms  of  simulated  run  time,  even  though  convergence  to  a 
limit  cycle  requires  up  to  10  cycles  (200ms).  The  repetition  rate  of  convergence 
from  one  pattern  to  the  next  in  the  model  exceeds  10  patterns/s. 
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